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Project  Abstract 

The  goal  of  this  project  is  to  fully  develop  Banach  space  methods  for  kernel-based 
machine  learning  that  extend  the  Hilbert  space  framework  of  regularized  learning. 
We  proposed  to  study  Reproducing  Kernel  Banach  Spaces  (RKBS)  by  the  semi- 
inner-product,  develop  the  theory  of  vector-valued  RKBS  with  applications  of  RKBS 
to  manifold  learning,  study  frames  and  Riesz  bases  for  sequence  spaces,  and 
construct  RKBS  with  the  I^-norm  known  to  enforce  sparse  solutions.  We  will  also 
explore  classification  algorithms  that  are  mathematically  rigorous  while  rooted  in 
human  cognitive  principles  for  categorization.  Our  execution  plan  include  three 
specific  topics  ("Aims")  1.  Apply  RKBS  theory  to  Orlicz  space,  to  perform 
convergence  analysis,  and  to  study  Shannon  sampling  schemes;  2.  Work  out  vector¬ 
valued  RKBS,  and  study  s.i.p  with  I^  norm;  3.  Develop  frames  and  Riesz  bases  for 
Banach  spaces,  and  extend  analysis  and  synthesis  operators. 

Accomplishments 

1.  Motivated  by  multi-task  machine  learning  with  Banach  spaces,  we  introduced  the 
notion  of  vector-valued  reproducing  kernel  Banach  spaces  (RKBS),  and  investigated 
basic  properties  of  the  spaces  and  the  associated  reproducing  kernels.  As  a 
highlight,  we  proved  that  the  kernel  K(x,y)  in  a  vector-valued  RKBS  is  a  bounded 
operator  satisfying  K(x,y)=(&(x)(&''’(y)  where  <&()  is  an  operator- valued  feature  map, 
with  (D^O  its  generalized  adjoint,  andx,y  are  sample  points.  This  is  an  extension  of 
the  feature  map  in  (scalar-valued)  RKBS  case.  Several  concrete  examples  of  the 
construction  of  the  feature  map  in  vector-valued  RKBS  were  presented.  Applying 
our  framework  to  multi-task  learning,  we  established  the  representer  theorem  and 
characterization  equations  for  the  minimizer  of  regularized  learning  schemes  in 
vector-valued  RKBS. 


2.  Frame  is  an  extension  of  the  notion  of  base  in  a  vector  space,  by  allowing 
dependence  among  members  of  a  collection  of  vectors.  A  Riesz  base  is  the  image, 
under  an  invertible  linear  transformation,  of  orthonormal  base  in  a  Hilbert  space, 
space.  Traditionally,  when  extending  those  concepts  to  the  Banach  space  B,  their 
definitions  involve  elements  of  the  dual  space  B*.  We  used  the  notion  of  semi-inner- 
product  and  propose  to  define  them  as  a  collection  of  elements  in  B  directly. 
Generalizing  the  classical  theory  of  Hilbert  space  frames  and  Riecz  basis  under  this 
new  perspective,  we  then  established  a  corresponding  Shannon  sampling  theorem 
in  Banach  spaces. 

3.  We  characterized  Orlicz  space  semi-inner  product  reproducing  kernel.  We  also 
derived  the  s.i.p  with  l^'l  norm,  and  made  progress  about  constructing  reproducing 
kernels  l^'l  norm  based  on  apriori  given  kernel  function,  but  a  satisfactory 
application  of  theory  to  derive  s.i.p  reproducing  kernel  remains  a  challenge. 

4.  Based  on  compressed  sensing  framework,  we  were  able  to  propose  and  construct 
a  low-rate  spiking  neuron  which  exploits  the  sparsity  or  compressibility  present  in 
natural  signals.  Our  model  neuron  belongs  to  the  class  of  lAF  (Integrate-and-Fire) 
neuron  model;  however  we  provided  appropriate  modifications  to  its  dead-time 
("absolute  refractory  period"  in  neuronal  term)  to  convert  it  into  a  low-rate  encoder, 
and  develop  an  algorithm  for  reconstructing/decoding  the  input  stimulus  from  the 
low-rate  spike  trains.  Our  neuron  produces  spikes  at  a  firing  rate  proportional  to  the 
amount  of  information  present  in  the  signal  rather  than  its  duration.  Through 
simulations  with  frequency-sparse  signals,  we  demonstrated  superior  performance 
of  our  Low-Rate  lAF  neuron  operating  at  a  sub-Nyquist  rate,  when  compared  with 
state-of-the-art  lAF  neurons  proposed  earlier  (e.g.,  A.  Lazar’s  Time  Encoding 
Machines  or  TEMs),  which  operate  at  and  above  Nyquist  rates. 

5.  Another  direction  (not  in  the  original  plan)  that  we  have  worked  and  made  major 
progress  is  a  formal  framework  to  unifying  feature  learning  and  generalization  in 
cross-table  with  a  list  of  objects  (in  rows)  and  a  list  of  features  (in  columns)  and 
derive  the  so-called  "concept  lattice"  -  a  lattice  is  a  partially-ordered  set  closed  in 
"meet"  and  "join"  operations.  We  are  able  to  reformulate  this  existing  framework  as 
"subset  system",  or  a  system  of  subsets  on  a  set.  For  any  subset  system,  we  show 
that,  in  addition  to  the  two  natural  relations  of  subsets  involved  (namely,  inclusion 
and  non-zero  intersection),  there  are  two  induced  relations  on  elements  of  the  base- 
set(which  model,  respectively,  *similarity*  by  a  symmetric  but  non-transitive 
relation  and  *specificity*  by  a  transitive  and  asymmetric  pre-order  relation).  We  can 
define  "neighborhood"  and  "separation"  in  ways  extending  those  as  used  in  abstract 
topology.  This  provides  a  mathematically  concise  way  of  modeling  "context"  in  a 
relational  data. 

Training  Activity 

Finally,  during  the  summer  of  2015,  the  PI  ran  a  summer  REU  (Research  Experience 
in  Undergraduate)  program  where  advanced  undergraduate  students  were  brought 


to  University  of  Michigan  for  2-8  weeks  to  engage  research  projects  conducted  by 
the  M3  Lab:  Alan  Aw  (Stanford  University),  Mark  Greenfield  (Caltech),  Luciana  Xiao 
(University  of  Nortre  Dame),  Bradley  Zykoski  (University  of  Virginia),  Michael  Lewis 
(New  Mexico  Institute  of  Mining  Technology).  This  was  the  first  time  the  PI  engages 
undergraduate  students  in  scale,  and  was  a  valuable  experience  in  training  and 
exposing  research  questions  to  the  next  generation  of  mathematician  and  scientists. 


